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Summary 


Every 26 seconds in the United States, a teenager drops out of school. This dire statistic, 
from a report by America’s Promise Alliance (Swanson, 2008), captured national attention 
and directed research and resources to tackling the nation’s dropout epidemic. Five years 
later, high school graduation outcomes continue to cause concern, and “large ‘graduation 
gaps’ remain in many states among students of different races, ethnicities, family incomes, 
disability, and limited English proficiencies” (Balfanz, Bridgeland, Bruce, & Fox, 2013, p. 
1 ). 

Many middle and high school principals express concern over students who unexpectedly 
begin to struggle and appear to fall off track without apparent reason. These educators 
want tools to help them identify these students as early as possible. Other school leaders 
believe they can tell, early in a student’s academic career, whether the student is at serious 
risk of not graduating. For these school leaders, the problem is not identifying the students 
who are struggling; it is figuring out how to get these students into meaningful interven- 
tions while accommodating districts’ financial and time constraints. 

A well implemented early warning system can help school leaders address these challenges. 
Early warning systems are used by states and districts across the nation to identify off-track 
students in middle and high school and to design and assess interventions to keep them on 
track to graduate (see Allensworth & Easton, 2007; George Washington University, 2012; 
Therriault, Heppen, O’Cummings, Fryer, & Johnson, 2010; Therriault, O’Cummings, 
Heppen, Yerhot, & Scala, 2013). 

Despite the popularity of early warning systems, research on their implementation is 
sparse. Most of the research has focused on developing and validating early indicators 
(Allensworth & Easton, 2007; Balfanz, Herzog, & Mac Iver, 2007; Balfanz, Wang, & 
Byrnes, 2010; Brown University, 2012; Gurantz & Borsato, 2012; Uekawa, Merola, Fernan- 
dez, & Porowski, 2010; University of Chicago Consortium on Chicago School Research, 
2007a, b,c). Organizations that have worked extensively in this area (such as the Amer- 
ican Institutes for Research, the University of Chicago Consortium on Chicago School 
Research, and the Everyone Graduates Center) have focused on creating early warning 
system tools and producing reports on promising practices (Bruce, Bridgeland, Fox, & 
Balfanz, 2011; Johns Hopkins University, 2012; Nagaoka, 2013; Therriault et al., 2013). 

This guide summarizes what is known about early warning system implementation and 
describes how states, districts, and schools can draw on the research to inform their work 
locally. 
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Why this guide? 


This guide describes the core ideas, recommendations, and experiences of districts and 
schools implementing early warning systems and provides examples of practices in use 
across the country. 1 The guide focuses on a key challenge identified in the literature: how 
to most productively access, interpret, and use early warning data. 


The guide discusses the following five core components of early warning system implementation: 

• Establishing and training a team to use the early warning system. 

• Identifying accurate indicators. 

• Designing and using reports. 

• Mapping appropriate interventions to individual student needs. 

• Evaluating student progress and intervention effectiveness. 

This guide is designed for state-, district-, or school-level practitioners who have some 
familiarity with early warning systems but want to learn more about their implementation. 
It may also help early warning system developers create systems that respond to practi- 
tioners’ needs and guide district- and school-level teams in implementing early warning 
systems more effectively. 2 Box 1 briefly explains early warning systems and the challenges 
associated with them. 


Box 1. Early warning systems: Arming districts and schools with data that can 
make a difference 


This guide is 
designed for state-, 
district-, or school- 
level practitioners 
who have some 
familiarity with 
early warning 
systems but 
want to learn 
more about their 
implementation 


Early warning systems use individual student data to generate indicators of on-track status 
for graduation, including attendance, behavior, and course performance (for more information, 
see box 3 on the ABCs later in this report). When a student falls below the on-track threshold, 
a team of school staff assigns the student to an intervention designed to help the student 
improve his or her performance. By systematically examining data, early warning systems can 
empower districts and schools to: 

• Identify struggling students earlier in their school career. 

• Direct students to appropriate interventions. 

• Examine and address on-track patterns among groups of students regularly. 

There are two major challenges to early warning systems: creating the system and helping 
staff members use it routinely. 

First, from a technical perspective, indicators must be valid and identify unique leverage 
points for intervention. While this may seem obvious, indicators and thresholds are often 
adopted without verifying their predictive power in their local context. Many districts do not 
have the resources to rigorously test the validity of each indicator, but most can at least 
examine previous years of student data for indicator trends relating to student performance 
and graduation outcomes. For a more detailed explanation, see the section “Ensuring local 
validity and threshold checking’’ in this guide. 

Second, from a practical perspective, indicator systems are not likely to produce the 
desired improvements in student outcomes unless they are part of a regular cycle of data use 
by teachers and principals (Therriault et al., 2013). 

Implementing an effective early warning system requires training and supporting local 
practitioners to use multiple types of data to understand student progress and difficulties. 
Such training and support should highlight how practitioners can apply their findings to pro- 
grams and strategies to improve graduation outcomes and college readiness and success. 
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Establishing and training a team to use the early warning system 


Implementing early warning systems effectively requires careful attention to team roles and 
responsibilities. Team structure, composition, leadership, goals, and community support 
should be determined in accord with district and school needs. 

Developing the team 

Before districts and schools begin to use an early warning system, they must establish a 
dedicated team of staff who will work together to identify off-track students, assign inter- 
ventions, and monitor progress. Typically, the teams are formed at the district or school 
level, depending on the number of students they serve. 

The Sioux Falls School District in South Dakota provides an example of a team convened 
solely at the district level. This team “include[s] four teachers, two school counselors, a 
school social worker, three elementary principals, two middle school principals, four cur- 
riculum services and special services administrators, and two instructional support services 
administrators” (Hauser & Koenig, 2011, p. 71). 

The Houston (Texas) Independent School District’s Dropout Recovery, Intervention, and 
Prevention Committees are teams convened at the school level, relatively independent of 
the district. On these teams, school faculty and staff, such as counselors and administra- 
tors, coordinate efforts with other community members, including police officers: “Every 
middle and high school has a Dropout Recovery, Intervention, and Prevention Committee 
made up of assistant principals, teachers, college access coordinators, clerks, police offi- 
cers, and counselors that meet weekly to monitor each student’s attendance and academic 
performance and to create and monitor ‘personal graduation plans’ for students who are 
identified as off track” (Houston Independent School District, 2012, p. 5). 

Another option is a mixed-level team, in which district- and school-level teams collabo- 
rate on responsibilities. The National High School Center suggests this approach, with 
school teams meeting more frequently than the district team. 3 The center recommends 
that district team members attend school-level team meetings and that a representative 
from each school team participate in the district-level team meetings. The center also rec- 
ommends including representation from each partner middle school outside the district 
that transitions students into the district’s high schools. Participation of all stakeholders 
who will interact with the early warning system encourages proper feedback cycles and can 
help early warning system teams coordinate and communicate more effectively (Therriault 
et al., 2013). 

Regardless of a team’s composition, it is imperative to establish good communication 
among all stakeholders, including district- and school-level leaders, administrative staff, 
counselors, teachers, information technology specialists, evaluation and assessment per- 
sonnel (where available), students, parents, and community partner organizations. Open 
communication about the early warning system’s purpose and use can help mitigate 
anxiety. 


Before districts 
and schools begin 
to use an early 
warning system, 
they must establish 
a dedicated team 
to identify off-track 
students, assign 
interventions, and 
monitor progress 
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Assigning roles and responsibilities 


Building an early warning system team usually starts with appointing a team leader who 
will facilitate meetings, check on data entry and updates, and oversee progress monitoring 
for individual students and intervention programs. Many early warning system teams also 
include data coaches who teach users how to access and interpret the data. A data coach 
from outside the school may serve as the team leader if external funding or partners are 
involved. The team leader can also be someone from within the school who has a strong 
understanding of the data and their potential use. It is helpful to include other key staff 
members and stakeholders (table 1). 


Defining goals and objectives 

Early warning system teams focus on reducing dropouts and increasing graduation rates. 
However, team members may view the team’s specific mission differently when they first 
meet. An initial goahsetting process can ensure that all members understand the team 
mission and help establish a road map for how the team will accomplish its goals. The 
S.M.A.R.T. framework can help the team design actionable, realistic objectives and time- 
frames (box 2) (Kekahio & Baker, 2013). 


Table 1. Common early warning system positions and responsibilities 

1 Position 

Responsibility 1 

Information technology staff 

Inputs data, collaborates on report structures, builds 
reports, updates data 

School team leader 

Serves as a liaison for the school with other schools and 
the district, ensures compliance with overall mission 

District-level leader 

Disseminates accomplishments and challenges, 
advocates for policy change at the district level 

Program coordinator 

Keeps track of the mapping between indicators and 
interventions available 

School counselor 

Represents the students' voice, relays what is working 
on a day-to-day basis 

Data coach 

Teaches the team how to interpret the data, identifies 
appropriate professional development 

Representatives from different stakeholder groups 
(for example, content-area teachers, special 
education and English language learner teachers, 
equity coordinators, social workers, and members 
of parent/student organizations) 

Serve as advisors and provide insight into how the 
system is working for a variety of stakeholders (some 
systems employ members on a rotating basis) 

Source: Authors' summary. 


The S.M.A.R.T. 
goal framework 
can help the 
early warning 
system team 
design actionable, 
realistic objectives 
and timeframes 


Box 2. Creating S.M.A.R.T. goals 

• S — Is the goal specific? (What will it do? Who will carry it out?) 

• M — Is the goal measurable? (How will the team know it has been achieved?) 

• A — Is the goal achievable? 

• R — Is the goal relevant to performance expectations? 

• T — Is the goal time bound? (How often will this task be done? By when will this goal be 
accomplished?) 

Source: Kekahio & Baker, 2013. 
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Determining meeting content and structures 


While schooHevel team meetings might periodically include discussions of aggregated indi- 
cator and intervention data, these meetings typically focus on individual students. Johns 
Hopkins University School of Education, Center for Social Organization of Schools (2012) 
provides extensive guidance on structuring the meetings of early warning system teams. 
A meeting may begin with a few minutes of announcements, include a brief discussion 
of students with time-sensitive issues (for example, pregnancy, major behavior violations), 
and then focus on students flagged as off track. The team would spend time discussing 
individual students (typically 5-10 minutes for each student). For new students this could 
include a debrief on the student’s specific challenges and flagged indicators; a discussion of 
staff members and interventions that could best help the student; and the assignment of 
a staff person, intervention, and follow-up date for tracking the student’s progress. For stu- 
dents previously flagged, the discussion could include a debrief on any changes in student 
status, an update on the student’s progress in the assigned interventions, and a discussion 
of next steps based on what seems to be working and what still needs attention. 

In a mixed-level team, meetings are more likely to include discussion about each school’s 
trends and an overview of interventions provided throughout the district. After the team 
collects and evaluates student progress and specific interventions, the district can review 
the interventions and schools that are progressing most effectively. 

It is up to the early warning system team to determine the frequency of meetings that is 
most appropriate for the district or school it oversees. In some cases, such as within the 
Houston Independent School District, school teams meet weekly to create intervention 
plans and discuss student progress (Houston Independent School District, 2012). School 
teams in the Big Brothers Big Sisters of Eastern Missouri, ABC Today! program also meet 
weekly. In addition, they convene monthly meetings with school principals and quarter- 
ly meetings with district leaders to update administrators and involve them in reviewing 
student data, trends, and outcomes (Bruce et al., 2011). 


Early warning 
system 

implementation 
frequently includes 
professional 
development 
for teachers, 
administrators, 
and other school 
staff in accessing 
the data and 
appropriately 
interpreting 
its meaning 


By contrast, the school transformation facilitator in the Diplomas Now program “facilitates 
biweekly early warning system meetings in which teams of teachers who share common 
groups of students and other student support staff members work together to interpret 
the data and design and plan effective interventions” (Herzog, Davis, & Legters, 2012, 
p. 4). The frequency of meetings should align with the school’s specific needs and structure 
(depending, for example, on the most commonly triggered indicators and the frequency 
of data updates). Even when there are no new data to discuss, team members may meet to 
review the status of student interventions or determine whether more training is required. 


Providing professional development for using the system 

Early warning system implementation frequently includes professional development for 
teachers, administrators, and other school staff in accessing the data and appropriately 
interpreting its meaning. Professional development can also help generate excitement 
around using the early warning system, shifting views from the system as “yet another 
task” to complete. 
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The Sioux Falls School District early warning system committee established a list of action 
initiatives to teach staff how to interpret early warning system results (Hauser & Koenig, 
2011, p. 71). The action initiatives include activities such as: 

• Training student assistance teams in a response-to-intervention model that uses 
universal, targeted, and individualized interventions in a tiered approach. 

• Training staff members to recognize risk factor subgroups and related triggers for 
intervention. 

Professional development can also be organized by the data coach. For example, Metro 
Nashville Public Schools used Race to the Top funding to employ an early warning system 
coordinator to oversee the districtwide program and 12 data coaches, one for each school 
cluster. The data coaches went to the schools “to build a culture among teachers and coun- 
selors for understanding student-, classroom-, and school-level early warning system data, 
and [for using the data] to guide intervention efforts” (Bruce et al., 2011, p. 28). 

Similarly, the Alabama State Department of Education provides 25 state-trained data 
coaches who are deployed to schools identified as “high-need” by the state’s accountability 
system (Bruce et al, 2011). The state also conducts professional development sessions on 
coaching students for graduation. Staff apply tools they learn in the sessions to guide their 
early warning system efforts. 

Seeking support through community volunteers 

Districts and schools often report feeling overwhelmed by the staff and financial resources 
needed to effectively implement an early warning system (Herzog et al, 2012). Teams can 
bolster their resources by partnering with community organizations. In the ABC Today! 
Program, Big Brothers Big Sisters of Eastern Missouri provided a “director of impact,” who 
was placed in five schools in the St. Louis Public Schools district to facilitate meetings and 
lead the early warning system team (Bruce et al, 2011). The program’s success inspired 
Wells Fargo Advisors to contribute financial and technical resources to build a data trans- 
fer system that increased the organization’s capacity to identify and intervene for students 
who were off track. 4 

The Diplomas Now program partners with the organization Communities in Schools, 
which provides a school transformation facilitator to help schools with their early warning 
system implementation (Herzog et al., 2012). This full-time position compiles data and 
facilitates biweekly early warning system meetings. Diplomas Now also partners with City 
Year, which provides AmeriCorps members to serve as full-time tutors, mentors, and role 
models for students identified through the early warning system. 

Identifying accurate indicators 


The primary function of an early warning system is to alert school community members 
(educators, parents, and students) when a student falls off track. System indicators should 
therefore include the strongest predictors of high school graduation. The number of indi- 
cators a district or school uses will frequently vary, but researchers advise that picking too 
many can create an overwhelming amount of data to analyze and interpret (University of 
Chicago Consortium on Chicago School Research, 2014), 


Early warning 
system teams 
can bolster 
their staff and 
financial resources 
by partnering 
with community 
organizations 
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Choosing indicators 


According to the University of Chicago Consortium on Chicago School Research (2014), 
effective indicators are: 

• Valid for the intended purpose. 

• Actionable by schools. 

• Meaningful and easily understood. 

• Aligned with district and school priorities. 


Consortium researchers advise early warning system teams to carefully question whether 
each of their proposed indicators meets these criteria. According to the consortium, 
student characteristics outside the school’s control, such as family income, special edit- 
cation status, and new student status, should not be used as indicators, despite research 
showing strong correlations between these variables and on-time graduation. 

Focusing attention on a small set of indicators allows early warning system teams to allo- 
cate their time and effort more efficiently. Many districts use some combination of atten- 
dance, behavior incidents, and course performance as a starting point for their dropout 
indicators (box 3). Early warning system teams can establish a base set of indicators first 
and then add other indicators. Over time, teams can assess whether the additional indica- 
tors substantially increase the number of students identified as off track. Unhelpful indica- 
tors can be dropped. Remember that each new indicator requires changing report formats, 
analyzing more data, and designing additional interventions to help the student improve 
in the indicator area. 


Focusing attention 
on a small set of 
indicators allows 
early warning 
system teams 
to allocate their 
time and effort 
more efficiently 


Although researchers at the Everyone Graduates Center and the National High School 
Center do not recommend using state assessment scores as an indicator, state-level early 
warning systems tend to include them because they correlate strongly with on-time 


Box 3. The ABCs 

The literature recommends that districts start with the ABCs as their base set of indicators. 

The ABCs stand for: 

• A — Attendance. Research has long shown a strong relationship between how often a 
student misses school and his or her probability of graduating in four years. Most students 
who do not regularly attend class fall behind in their coursework and consequently see 
their grades suffer. Poor attendance can also indicate that a student is struggling with 
health, family, or other issues that are distracting them from their studies. 

• B — Behavior incidents. As few as one suspension in grade 6 may predict whether a 
student graduates in four years (Balfanz et al., 2007). Behavior incidents can indicate 
that a student is disengaged with the school environment. Suspensions often cause an 
additional burden on students to catch up on the material they missed. 

• C — Course performance. Number of course failures and overall grade point average obvious- 
ly correlate with a student’s probability of graduating in four years. If a student fails a course, 
he or she will need to make up the credit outside the regularly scheduled school time to stay 
on track. Poor course performance can also indicate disengagement at the classroom level. 

Note: For more information on each indicator (as cited) see Allensworth and Easton (2007) (A,C); Balfanz et al. 

(2007) (A,B,C); Balfanz et al. (2010) (A,B,C); Celio (2009a, b) (A,B,C); Mac Iver (2010) (A,B,C); Mac Iver and 

Mac Iver (2009) (A,B,C); Roderick (1993) (A,C); Uekawa et al. (2010) (A,C). 
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Table 2. Massachusetts academic goals by grade level 

Risk model 
age group 

Grade level 

Expected student outcome for each grade group 

Early elementary 

Grades 1-3 

Proficient or advanced on grade 3 Massachusetts 
Comprehensive Assessment System English language arts 
assessment (English state test scores) 

Late elementary 

Grades 4-6 

Proficient or advanced on grade 6 Massachusetts 
Comprehensive Assessment System English language arts 
assessment and math assessment (English and math state 
test scores) 

Middle grades 

Grades 7-9 

Passing grades on all grade 9 courses 

High school 

Grades 10-12 

High school graduation 


Source: Summary based on information from Massachusetts Department of Elementary & Secondary Educa- 
tion (2013a). 


graduation. Early warning system teams should consider the purpose of their system when 
deciding whether to include state or other assessment scores. If the system is designed to 
monitor student progress at more regular intervals, these scores may be less appropriate. 

Systems that include elementary grade reporting, however, may want to use state assess- 
ment scores to set on-track goals for younger children since labeling elementary students as 
potential dropouts may be considered premature. Massachusetts, for example, has identi- 
fied unique academic goals for each grade level, which are then used to determine whether 
a student is on track to graduate (table 2). 


The literature 
advises early 
warning system 
teams to set 
thresholds 
for indicators 
using local data 
wherever possible 


Ensuring local validity and threshold checking 

The literature advises early warning system teams to set thresholds for indicators using 
local data wherever possible. Indicators can have degrees of reliability that vary by context 
and the precise definition a district or school uses for a given measure. For example, some 
schools may consider a student absent if he or she misses one class period. Others may 
have more lenient policies that do not count an absence until the student misses more 
than half a school day. 

Teams can set threshold levels by beginning with recommendations from national orga- 
nizations that have extensive experience with early warning systems such as the National 
High School Center and Johns Hopkins University. The National High School Center 
suggests thresholds to create a two-tiered system for identifying students as on or off track 
for graduation (table 3). John Hopkins University provides suggestions for establishing 


Table 3. National High School Center’s suggested early warning system indicators 
and thresholds 

1 Indicator 

Threshold A 

Attendance 

Student missed 10 percent or more of instructional time (absences) 

Behavior 

Locally validated thresholds (for example, referrals, in- or out-of-school 
suspension, behavior grades) 

Course performance 

Failure in one or more courses 

Earned 2.0 or lower grade point average (on a 4-point scale) 

Source: Therriault et al. 

, 2013, p. 2. 
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Table 4. Johns Hopkins University’s suggested early warning system indicators and thresholds 



Attendance 


Behavior 





(days missed) 

Office referrals 

Suspensions 

Academics 

Threshold 

Quarter 

Full year 

Quarter 

Full year 

Quarter 

Full year 

Math and English 
language arts a 

Core courses ' 1 

Off track 

9 days 

36 days 

2 

6 

1 

2 

Report card grade 
of F 

Report card grade 
of F 

Sliding 

5-8 days 

19-35 days 

1 

3-5 

0 

0-1 

Report card grade 
of D 

Report card grade 
of D 

On track to 
graduation 

4 days 
or fewer 

18 days 

0 

0-2 

0 

0-1 

Report card grade 
of A, B, or C 

Report card grade 
of A, B, or C 

a. Middle school. 









b. High school. 










Source: Johns Hopkins University, 2012, p. 10. 


thresholds using a three-tiered system to identify students as off track, sliding, or on track 
to graduation (table 4)- Teams can then experiment with their historical data to see 
whether changing thresholds may make sense in their context. 

Early warning system teams can determine the accuracy of thresholds by examining his- 
torical data on the number of students who dropped out in the past three to five years who 
would have been identified through the system. 5 For example, if the team decided that 
two days of unexcused absences would serve as an indicator that a student is at risk but 
then found that the majority of such students actually graduated, the team might want to 
increase the number of unexcused absences to four and re-examine the data. Likewise, if 
many students who dropped out are not identified at four unexcused absences, the team 
might want to decrease the number to three. Teams can re-examine the accuracy of their 
indicators and thresholds each year as they acquire new data, since thresholds are likely 
to change. For example, if a team achieves positive results from targeting attendance, over 
time the team will most likely need to set higher expectations for student attendance. 
Research suggests that the best thresholds flag a substantial number of students who can 
be helped through interventions, without overwhelming the capacity of the early warning 
system team to respond (Balfanz et al., 2010; Hauser & Koenig, 2011). 6 


Once thresholds 
for each indicator 
are finalized, the 
team can begin 
placing students 
into categories 
(for example, “on 
track," “sliding," 
and “off track”) 


Once thresholds for each indicator are finalized, the team can begin placing students into 
one of three categories: “on track” (no perceived issues), “sliding” (at least one indicator 
shows that the student may be struggling), and “off track” (at least one indicator shows 
that the student, on his or her current trajectory, is at high risk of not graduating in four 
years). The categories can help the team direct students to appropriate interventions based 
on the intensity of assistance the student may require. 


Designing and using reports 


An early warning system focuses on two primary actions: compiling data that identify 
student progress toward graduation and alerting staff when the data indicate that a student 
is falling off track. Developing usable reports for district and school staff is therefore a 
crucial part of early warning system implementation. 
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Making reports simple but effective 


Early warning system teams should collaborate closely with their information technolo- 
gy departments (where available) to develop an easy-to-use reporting structure. 7 Teams 
should consider the different audiences they are targeting and the need for various report- 
ing formats. School summary reports may be most helpful for administrators, while class- 
room-level reports may provide more insight for teachers. Student-level reports may be 
useful for teachers and counselors who work with individual students. These reports could 
also be disseminated to the students and their families. 


For example, the University of Chicago Consortium on Chicago School Research 
(2007a, b,c) has created a series of “What Matters” briefs, targeted specifically to freshman 
students, parents, and teachers. These two-page flyers include charts to demonstrate how 
students’ grades and attendance can affect their chances of graduating from high school. 
They further explain the importance of establishing good habits to keep from falling 
off track. The flyers encourage students, parents, and teachers to share responsibility for 
helping the student graduate on time. Districts or schools implementing early warning 
systems could create a similar set of personalized briefs for students and families to illus- 
trate how the student’s performance compares with district and school averages and with 
early warning system thresholds. 


Regularly updating 
indicator data 
will ensure that 
the early warning 
system team 
can accurately 
monitor and track 
student progress 


Establishing a process for routinely creating and using reports 

The early warning system teams should update their indicator data regularly to reflect current 
student data. Regular updates will ensure that the early warning system team can identify stu- 
dents who have recently fallen off track, recognize gains for students who have improved, and 
reassess intervention strategies for students who were previously flagged and assigned to inter- 
ventions but remain off track. The team may discuss all students at each meeting or, if the 
number of off-track students is too large for the team to discuss in the time available, the team 
can divide students into smaller subsets assigned to different team members. If the second 
option is chosen, the team should still debrief as a group on actions taken and useful strategies. 
Team meetings will focus on using the reports to monitor and track student progress. 

The frequency of early warning system reporting updates will depend on data availability, 
which can vary across indicators. Some indicators, such as course grades, may be available 
for update only at certain intervals. For example, the Houston Independent School Dis- 
trict’s Dropout Prevention/Early Warning Report system inputs failing grades from prog- 
ress reports every three weeks (Fogan-Fain, 2011). Updates for other indicators, such as 
attendance and behavior, may be available more frequently. When considering how often 
to meet, early warning system teams should work with information technology staff to 
determine how frequently they can expect to receive new information about students. 

The student data needed to create the early warning system reports will contain personally 
identifiable information, which is covered by privacy laws. In addition to complying with 
federal laws, such as the Family Educational Rights and Privacy Act and the Health Insur- 
ance Portability and Accountability Act, observing student confidentiality is essential for 
protecting families from potentially embarrassing disclosures, discrimination, differential 
treatment, and threats to family and job security. Early warning system teams should consult 
with legal counsel about ensuring confidentiality for early warning system data and reporting. 
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Mapping appropriate interventions to individual student needs 


The fourth core component in successfully implementing an early warning system is to 
align intervention efforts with the chosen indicators. Creating a direct link between indi- 
cators and interventions is essential for getting students the individualized assistance they 
need to improve and, ultimately, graduate. 

Surveying what is available 

Districts deploy a variety of high school graduation strategies and interventions, ranging 
from districtwide policies to more individualized student programs (Herzog et al., 2012). 
Districts, therefore, may find it more productive to first catalogue all the interventions — the 
programs and policies currently in place to help students succeed in school — before adopt- 
ing new strategies. For instance, the Louisiana Dropout Early Warning System includes a 
list of more than 50 interventions to which early warning system teams can assign off-track 
students (Louisiana Department of Education, 2009). These interventions range from 
tutoring programs and a homework hotline to home visits or moving the student to an 
alternative site or school. Additionally, teams can check the What Works Clearinghouse 
(http://ies.ed.gov/ncee/wwc/findwhatworks.aspx) for lists of effective interventions with 
strong evidence of effectiveness specifically for improving outcomes in behavior, some aca- 
demic subjects, and, more generally, dropout prevention. The early warning system team 
may want to openly discuss and document any constraints with intervention programs 
or policies. Have they found the program to be successful? Is there a limit to the number 
of students enrolled in the program? Has the policy been difficult to implement? Can the 
team identify steps to monitor and improve implementation of the program or policy? 

Early warning system teams should first consider adjusting the initiatives and interventions 
they have already invested in. For example, could changing the math curriculum engage 
more students in math classes? Or could revising district or school attendance policies 
make a difference? However, teams may also need to implement new programs and policies 
to effectively intervene for identified students. 

Merging response-to-intervention efforts with early warning systems 

Districts considering or implementing an early warning system often also use a 
response-to-intervention model with a tiered approach to supporting student success. This 
model has three levels of intervention (American Institutes for Research, National Center 
on Response to Intervention, 2010, p. 4): 

• Primary level: Focuses on all students and involves evidence-based “district curric- 
ulum and instructional practices that . . . align with state or district standards and 
incorporate differentiated instruction.” 

• Secondary level: Focuses on identified at-risk students and provides “targeted, sup- 
plemental instruction delivered in small groups.” 

• Tertiary level: Focuses on students struggling after receiving primary and second- 
ary interventions and provides “intensive, supplemental instruction delivered indi- 
vidually or to small groups.” 8 

The response-to-intervention model can provide early warning system teams with a useful 
framework to conceptualize and manage their interventions. Likewise, early warning 
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Table 5. Comprehensive plan for keeping students on the graduation path 

Type of 


Focus of intervention 


intervention 

Attendance 

Behavior 

Course performance 

Schoolwide 
(all students) 

• Every absence brings a 
response 

• Create a culture that 
says attending every day 
matters 

• Positive social incentives 
for good attendance 

• Data tracking by teacher 
teams 

Teach, model, and expect 
good behavior 

• Positive social incentives 
and recognition for good 
behavior 

• Advisory 

- Data tracking by teacher 
teams 

• Research-based 
instructional programs 
In-classroom support 
to enable active and 
engaging pedagogies 

• Data tracking by teacher 
teams 

Targeted 
(15-20 percent 
of students) 

Two or more unexcused 
absences in a month 
brings brief daily check by 
an adult 

• Attendance team 
(teacher, counselor, 
administrator, parent) 
investigates and problem 
solves (why isn't student 
attending?) 

Two or more office 
referrals brings 
involvement of behavior 
team 

• Simple behavior checklist 
students bring from class 
to class, checked each 
day by an adult 

• Mentor assigned 

• Elective extra-help 
courses — tightly linked to 
core curriculum — preview 
upcoming lessons and fill 
in knowledge gaps 
Targeted, reduced class 
size for students whose 
failure is rooted in social 
or emotional issues 

Intensive 
(5-10 percent 
of students) 

• Sustained one-on-one 
attention and problem 
solving 

• Appropriate social 
service or community 
support 

• In-depth behavioral 
assessment (why is 
student misbehaving?) 

• Behavior contracts with 
family involvement 

• Appropriate social 
service or community 
supports 

• One-on-one tutoring 

Source: Mac Iver & Mac Iver, 2009, p. 23. 


Teams can easily 
survey their 
mapping for 
potential gaps 
in intervention 
availability 


systems can provide response-to-intervention models with stronger, more regularly avail- 
able data for student progress monitoring. Districts using response-to-intervention models 
might consider combining the efforts of these two initiatives. Mac Iver and Mac Iver 
(2009) describe how the two frameworks can map together in their comprehensive plan for 
keeping students on the graduation path (table 5). 9 

Mapping interventions to indicators 

The next step is to identify which indicator the intervention can best address. For example, 
afterschool tutoring may benefit students flagged because of course grades. A peer mediator 
group may help students with behavioral incidents. A spreadsheet program such as Excel 
can facilitate mapping by listing interventions as rows and indicators as column variables. 
A focused intervention is tied directly to a specific indicator; however, such interventions 
may improve performance on multiple indicators. Creating a structure that accommodates 
using multiple links can help simplify the process of identifying the programs best suited 
for flagged students. The sample intervention-indicator mapping shows how this can be 
done (table 6). 

Teams can easily survey their mapping for potential gaps in intervention availability. 
Where substantial gaps exist, the team may consider asking for additional resources to 
fund a new program or reallocating current funding from indicator areas where resources 
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Table 6. Sample intervention-indicator mapping 





Focus of intervention (ABCs) 

Intervention 

Notes 

Attendance Behavior 

Course performance 

Afterschool tutoring 

Available only for math 



✓ 

Peer mediators 

Accommodates 15 students 
per grade 


✓ 


Individual coaching 

We have four coaches who can 
each manage 10 students 

✓ 

✓ 

✓ 

First period check-in 


✓ 



Policy to talk to parents 
after two unexcused 
absences 

Do we have staff available 
to personally talk to every 
parent? 

✓ 



Source: Authors' summary. 


are available. If new funds are not available and existing programs are already spread too 
thin, the team may consider approaching community or business partners to help close the 

gap- 

Early warning system teams should closely monitor their systems to observe when an inter- 
vention is not meeting a student’s needs, based on the intervention’s level of intensity and 
the length of time it should take a typical student to improve. For example, if the student 
attends a daily intervention rather than a monthly support group, the team might expect 
faster results. Teams can examine a student’s indicators for the anticipated change. After 
a predetermined amount of student participation time, the team can then meet with the 
student to see how well the intervention is working. 


The early warning 
system team may 
investigate a 
student’s needs 
by talking to the 
student directly 
and conferring 
with the student’s 
family and other 
school staff 
to determine 
appropriate 
interventions 


Assigning interventions to students 

As noted earlier, when an early warning system team meets, its members will discuss the 
students who are off track and determine interventions for these students. These inter- 
ventions should correspond to the flagged indicators and to the individual student’s char- 
acteristics and needs. For example, in the Sioux Falls School District all students who 
are flagged as off track are assigned a student engagement case manager (for example, a 
teacher, counselor, or success coordinator) who interacts with the student daily. Further 
interventions depend on the indicator. Students who are flagged for the academic indica- 
tor are assigned to learning centers or afterschool tutoring for additional study time, while 
a student flagged for having more than 10 absences is assigned a social worker or counselor 
advocate who investigates the cause of the student’s poor attendance (Hauser & Koenig, 
2011). 

The early warning system team may investigate a student’s needs by talking to the student 
directly and conferring with the student’s family and other school staff (such as teach- 
ers, coaches, counselors, and administrators); all may be able to provide insights into why 
the student is off track. The team can then determine the appropriate interventions and 
monitor their frequency based on the student’s needs and the school context. 

Teams are encouraged to consult resources such as the What Works Clearinghouse to iden- 
tify interventions with strong evidence of effectively improving student outcomes. Teams 
may also consider partnering with organizations outside their district or school to gain 
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further insight into why certain students are struggling. Mentorship is used in the Dipk> 
mas Now program, which provides near-peer mentors from City Year (a national service 
program). Diplomas Now also provides “social workers/site coordinators from Communities 
in Schools . . . and a school transformation facilitator ... [to provide] both insight into the 
challenges students face, and additional person power to help implement and monitor the 
proposed solutions” (Bruce et al., 2011, p. 32). 

Evaluating student progress and intervention effectiveness 


To maximize the benefits of an early warning system, districts and schools need to know 
whether student performance improves after an intervention is put in place and whether 
they need to adopt new or different strategies to address gaps in student supports. Prop- 
erly implementing an early warning system and carefully recording student progress by 
intervention can help teams evaluate how well programs and policies are meeting student 
needs. Team members can also use these data to investigate whether these trends are corn 
sistent across subgroups of students. 

Examining student progress 

Key student data to track will include, at a minimum, the interventions prescribed for the 
student. However, to improve the process of examining individual interventions, teams 
should also collect data on how often the student has participated in each intervention 
and the student’s performance in the indicators on which he or she was flagged. 

Recording these variables will allow the team or case manager to examine the data over 
time. It is crucial for the early warning system team to review this information to track the 
student’s progress and modify the intervention plan if the student remains off track. The 
team should determine how frequently it can expect updates on student progress data, in 
the same way it receives updates to the early warning system data. 

Ideally, the early warning system database will provide fields to collect this information; 
however, many systems do not incorporate this functionality. The early warning system 
teams will thus need to record the information themselves, using an Excel spreadsheet 
or Access database, coordinate the effort with their information technology staff, or use 
other resources. For example, because the Massachusetts Early Warning Indicator System 
does not allow for recording interventions internally, the state’s Department of Elementary 
& Secondary Education suggests that “schools and districts consider using the seven-step 
process created by the National High School Center to analyze Early Warning Indicator 
System data, assign student interventions, and review student progress” (Massachusetts 
Department of Elementary & Secondary Education, 2013b, p. 17). 10 

Examining the intervention level 

One important reason districts and schools might consider examining their own inter- 
ventions for student outcome trends is the scarcity of rigorous studies on the effectiveness 
of early warning system intervention strategies. Even when considering less rigorous eval- 
uations, little information is available on the effectiveness of either interventions or early 
warning systems as a whole. One factor may be that districts and schools have only recent- 
ly adopted these systems, so the research community may not have had time to conduct 
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evaluations. Also, until recently, early warning system implementation has not emphasized 
the careful recording of student progress. Without accurate data on how students respond 
to interventions, internal or external research partners cannot analyze the impacts of their 
programs. 


To ensure better and more accurate data on interventions, early warning system teams are 
encouraged to include procedures for recording student progress data in their planning and 
schedule time to discuss intervention data among their team and with other district and 
school leaders. Properly recording student progress by intervention can help district and 
school teams examine trends in how well programs and policies are meeting student needs 
and what emerging needs may require greater support. Accurate data also enable future 
researchers to evaluate more rigorously select interventions or programs and thus provide 
information that can help teams learn from their experience. 

During early warning system meetings, for example, the team can monitor and examine 
trends in student outcomes for each intervention strategy. The Louisiana Dropout Early 
Warning System documentation states that beyond identifying students and assigning 
interventions, the system aims to record and track success for developing best practices. 
This system includes preset reports to encourage this goal. Within a given intervention 
the system displays a line or bar chart showing the number of students who received the 
intervention each month. The system also compares across intervention types through a 
table whose cells display the number of students who triggered a given indicator and were 
assigned to each intervention (Louisiana Department of Education, 2009). 


Early warning 
system teams can 
use intervention 
data along with 
individual student 
reports to see 
which interventions 
are useful 


Although tracking intervention data is no substitute for a rigorous evaluation, early 
warning system teams can use these data along with individual student reports to see 
which interventions are useful and determine whether usefulness differs by indicator. 
Intervention data are frequently examined annually (Therriault et al., 2013), though teams 
are advised to review their data as needed based on their specific district and school con- 
texts and the expected rate of student improvement. 

Implications of this guide 


This guide summarizes what is currently known about early warning system implementa- 
tion and aims to help those who are actively working to implement these systems in their 
states, districts, and schools. Research continually suggests that early warning systems will 
not succeed merely by providing data (Johns Hopkins University, 2012, p. 30; Therriault 
et al., 2013, pp. 6-9). Districts and schools that report the most progress with their early 
warning systems have paid careful attention to systemdevel implementation, actively 
endorsing and supporting their early warning systems across all levels of leadership and 
providing professional development to all system users. Much forethought has also gone 
into the roles each team member will play and the data they will collect and record. Early 
warning systems should not be used alone, but as part of a framework to incorporate data 
use into decisionmaking. And early warning systems can be expanded to cover other grade 
levels or college and career readiness (box 4). 

Teams can adapt indicators and thresholds to their local contexts and carefully align 
interventions to create a comprehensive net to catch students who might otherwise have 
been overlooked. However, even accurate thresholds will not identify every student at risk 
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Box 4. Other early warning system applications 


While early warning systems are most prevalent in high schools, some systems cover middle 
school grades or earlier. FHI 360’s Indicators for Success: Interventions and Supports program 
(http://middlestart.org/network/group/early_indicators/, retrieved April 25, 2013), implement- 
ed in the New York City Department of Education, focuses on early warning system use in 
middle schools. The process outlined in this guide could easily be adapted for a middle school 
framework. 

The Minneapolis School District hopes to expand its program to address student needs 
as early as elementary school. “Minneapolis is analyzing the dropout data from its class of 
2010 to identify early warning signs of those students all the way back to grade 3, in an effort 
to implement Check and Connect interventions at earlier grades” (Sparks, 2011, p. 8). Going 
this far back could prove useful, though it is beyond the scope of most early warning systems. 

Some programs have expanded their early warning systems to include measures for 
determining progress toward college and career readiness. For example, Brown University’s 
Annenberg Institute for School Reform and Stanford University’s John W. Gardner Center have 
partnered to launch the College Readiness Indicator Systems initiative. Together they have 
worked with school districts in Dallas, New York City, Philadelphia, Pittsburgh, and San Jose to 
develop and implement expanded early warning systems. Beyond the usual tracking of atten- 
dance, behavior, and course performance, these systems monitor academic preparedness, 
academic tenacity, and college knowledge (Gurantz & Borsato, 2012). 

New Visions for Public Schools has partnered with College Readiness Indicator Systems in 
New York City to expand the traditional early warning system approach. In addition to looking 
at point-in-time “stocks” of students who are “off track,” “almost on track,” “on track to grad- 
uation," and “on track to college," New Visions maps out “flows” of students as they move up 
or down between these categories (Brown University, 2012). School administrators use this 
“flow map” to examine how students are doing, either within a cohort year or schoolwide. They 
then use these data to identify school structures or interventions that may be helping some 
students improve but inadvertently drawing resources away from other students and putting 
them at risk of dropping to a lower category. By identifying such feedback loops, New Visions 
strongly encourages administrators to see how the school functions as a system and make 
course corrections to prevent helping one group of students at the expense of another. 


of not graduating. An early warning system used in conjunction with other dropout pre- 
vention and school improvement strategies (for example, response-to-intervention models) 
will identify and support the greatest number of students at risk. By planning carefully and 
incorporating resources from the outset of early warning system adoption, districts and 
schools can more accurately identify potential dropouts and help keep these students on 
track for graduating with their peers. 
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Notes 


1. All suggestions provided in this guide are drawn from articles listed in the referenc- 
es. Key references are specifically identified in the guide summary. New citations are 
added if an idea, or quote, was specific to only one article or author. 

2. For a shorter summary of the information provided in this guide, see Frazelle and 
Barton (2013). 

3. The National High School Center’s seven year grant from the U.S. Department of 
Education’s Office of Elementary and Secondary Education and Office of Special Edu- 
cation Programs ended on March 31, 2013. Its website and resources remain available 
indefinitely to users through support from the American Institutes for Research, but 
content and external links have not been updated since then. 

4. Another example is the StriveTogether model being used in Cincinnati (see http:// 
www.strivetogether.org). 

5. For more specific recommendations from national organizations that have worked 
extensively with early warning systems and methods for setting indicator thresholds, 
see Balfanz et al. (2010, p. 5) and Hauser & Koenig (2011, pp. 25-42). 

6. One way to manage this process can be through using a response-to-intervention 
framework (see section “Merging response-to-intervention efforts with early warning 
systems” later in this guide). 

7. For example, reports from FHI 360’s Indicators for Success: Interventions and Supports 
program represent the on-track, sliding, and off-track categories through color coding 
(green, yellow, and red, respectively) to help practitioners identify students more easily. 

8. For more details and references on response-to-intervention implementation, see 
Mclnerney and Elledge (2013). 

9. Additional examples can be found in Bruce et al. (2011, pp. 20-23). 

10. For more information on this process, see http://www.betterhighschools.org/EWS_imp. 
asp. 


Notes-1 


References and resources 


Alabama State Department of Education. (2011). Introduction to the Graduation Tracking 
System (GTS). Montgomery, AL: Author. Retrieved January 20, 2014, from http://alex. 
state. al.us/discipline/sites/alex.state.al.us.discipline/files/GTS%20Imp%20Guide%20 
Final%208-23-ll.pdf 

Allensworth, E. M., & Easton, J. Q. (2007). What matters for staying on-track and graduating 
in Chicago public high schools: A close look at course grades, failures, and attendance in 
the freshman year. Chicago, IL: University of Chicago Consortium on Chicago School 
Research. http://eric.ed.gov/lkDED498350 

American Institutes for Research, National Center on Response to Intervention. (2010). 
Essential components ofRTI: A closer look at Response to Intervention. Washington, DC: 
Author. http://eric.ed.gov/lkUED526858 

Balfanz, R., Bridgeland, J. M., Bruce, M., & Fox, J. H. (2013). Building a grad nation: Progress 
and challenge in ending the high school dropout epidemic (Annual updated ed.). Baltimore, 
MD: Johns Hopkins University, School of Education, Everyone Graduates Center. 
http://eric.ed.gov/lkUED542115 

Balfanz, R., Herzog, L., & Mac Iver D. J. (2007). Preventing student disengagement and 
keeping students on the graduation path in urban middle-grades schools: Early idem 
tification and effective interventions. Educational Psychologist, 42 ( 4), 223-235. http:// 
eric.ed.gov/?kUEJ780922 

Balfanz, R., Wang, A., & Byrnes, V. (2010). Early warning indicator analysis: Tennessee. 
Nashville, TN: Tennessee Department of Education, Office of Safe & Supportive 
Schools. Retrieved January 10, 2014, from http://www.tn.gov/education/safe_schls/ 
dropout/doc/EarlyWarninglndicatorAnalysisTN.pdf 

Brown University, Annenberg Institute for School Reform. (2012). Beyond individual at-risk 
studentperformance: Anew tool to identify systemic causes ofriskinNew YorkCity highschools. 
Providence, RI: Author. Retrieved January 10, 2014, from http://annenberginstitute 
.org/pdf/CRIS_Webinar4.pdf 

Bruce, M., Bridgeland, J. M., Fox, J. H., & Balfanz, R. (2011). On track for success: The use 
of early warning indicator and intervention systems to build a grad nation. Baltimore, MD: 
Johns Hopkins University, School of Education, Everyone Graduates Center, http:// 
eric.ed.gov/?kUED526421 

Celio, M. B. (2009a, April). Getting to graduation: Kent School District cohort study. Pow- 
erPoint presentation to Kent School District. Retrieved January 10, 2014, from http:// 
www.roadmapproject.org/wp'Content/uploads/2012/08/EWIs_Kent'Schools'2009 
-Cohort'Study.pdf 

Celio, M. B. (2009b, September). Seattle School District 2006 cohort study. PowerPoint pre- 
sentation to Bill & Melinda Gates Foundation. Retrieved January 10, 2014, from http:// 


Ref-1 


www.roadmapproject.org/wp-content/uploads/2012/08/EWIs_Seattle-Public-Schools 

'2006'Cohort'Study.ppt 

Clemson University, National Dropout Prevention Center/Network. (2013). Model 
program strategies. Clemson, SC: Author. Retrieved January 10, 2014, from http://www. 
dropoutprevention.org/modelprograms/program-strategies 

Dynarski, M., Clarke, L., Cobb, B., Finn, J., Rumberger, R., and Smink, J. (2008). Dropout 
prevention: A practice guide (NCEE 2008-4025). Washington, DC: National Center for 
Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. 
Department of Education. Retrieved January 20, 2014, from http://ies.ed.gov/ncee/wwc 

Frazelle, S., & Barton, R. (2013). Implementing early warning systems. Principal’s Research 
Review, 8(4), 1-7. Retrieved January 10, 2014, from http://www.nassp.org/Content/158/ 
prr_julyl3_web.pdf 

George Washington University, Center for Equity & Excellence in Education. (2012). 
Evidence-based resources for keeping students on track to graduation. Washington, DC: 
Author. Retrieved January 10, 2014, from http://www.doe.virginia.gov/support/school_ 
improvement/titlel/1003_g/resources/evidence_based_resources.pdf 

Gurantz, O., & Borsato, G. N. (2012). Building and implementing a college readiness indi- 
cator system: Lessons from the first two years of the CRIS initiative. Voices in Urban 
Education, 35, 5-15. Retrieved January 10, 2014, from http://www.annenberginstitute. 
org/VUE/wp-content/pdf/VUE35_Gurantz 

Hauser, R. M., & Koenig, J. A. (Eds.). (2011). High school dropout, graduation, and com- 
pletion rates: Better data, better measures, better decisions. Washington, DC: National 
Academies Press. http://eric.ed.gov/lkDED536469 

Herzog, L., Davis, M., & Legters, N. (2012). Learning what it takes: An initial look at how 
schools are using early warning indicator data and collaborative response teams to keep 
all students on track to success. Baltimore, MD: Johns Hopkins University, School of 
Education, Everyone Graduates Center. Retrieved January 10, 2014, from http://new. 
everylgraduates.org/wp-content/uploads/2012/04/Learning_what_it_Takes.pdf 

Houston Independent School District. (2012). Apollo 20 [Program information and fact 
sheets]. Houston, TX: Author. Retrieved January 10, 2014, from http://www.senate. 
state.tx.us/75r/senate/commit/c530/handoutsl2/0913-RodneyWatson-2.pdf 

Hurwitch, B., & Stefanakos, M. (2011, July). Maine Statewide Longitudinal Data System 
phase 2. PowerPoint presentation at the STATS-DC 2011 NCES Data Confer- 
ence, Bethesda, MD. Retrieved January 10, 2014, from http://nces.ed.gov/whatsnew/ 
conferences/statsdc/2011/presentations/V_C.pdf 

Johns Hopkins University, School of Education, Center for Social Organization of 
Schools. (2012). Using data to keep all students on track to graduation: Team playbook. 
Baltimore, MD: Author. Retrieved January 10, 2014, from http://new.everylgraduates. 
org/wp-content/uploads/2012/ll/Team_Playbook.pdf 


Ref-2 


Kekahio, W., & Baker, M. (2013). Five steps for structuring data-informed conversations 
and action in education (REL 2013-001). Washington, DC: U.S. Department of 
Education, Institute of Education Sciences, National Center for Education Evalu- 
ation and Regional Assistance, Regional Educational Laboratory Pacific. http://eric. 
ed.gov/?id=ED544201 

Logan-Fain, L. (2011, November). Dropout prevention: A conversation of research-based 
recommendations and successful implementation. PowerPoint presentation at Edvance 
Research, Texas Consortium on School Research hosted webinar. Retrieved January 
10, 2014, from http://tcsr.edvanceresearch.com/downloads/TSCR%20Webinar%20 
Slidesll0711.pptx 

Louisiana Department of Education. (2009). Louisiana Dropout Early Warning System: 
DEWS from the ground up. Presentation at Data Quality Campaign hosted webinar. 
Retrieved January 10, 2014, from http://www.dataqualitycampaign.org/files/EWI_'_ 
DEWS'Presentation_Webinar_.pdf 

Mac Iver, M. A. (2010). Gradual disengagement: A portrait of the 2008—09 dropouts in 
the Baltimore City Schools. Baltimore, MD: Baltimore Education Research Consor- 
tium. Retrieved January 10, 2014, from http://baltimore'berc.org/pdfs/Gradual%20 
Disengagement%20final.pdf 

Mac Iver, M. A., & Mac Iver, D. J. (2009). Beyond the indicators: An integrated school-lev- 
el approach to dropout prevention. Washington, DC: George Washington University, 
Center for Equity and Excellence in Education, Mid-Atlantic Equity Center, http:// 
eric.ed.gov/?id=ED543512 

Mack, S. (2014, April 8). Putting student data to the test to identify struggling kids [Audio broad- 
cast & transcript]. Miami, FL: WLRN. Retrieved April 28, 2014, from http://www.npr. 
org/2014/04/08/300587823/putting'Student'data-tO'the'test'tO'identify'Struggling'kids 

Massachusetts Department of Elementary & Secondary Education. (2013a). Frequently 
asked questions. Boston, MA: Author. Retrieved January 10, 2014, from http://www. 
doe.mass.edu/edwin/faq.html 

Massachusetts Department of Elementary & Secondary Education. (2013b). Massachusetts 
Early Warning Indicator System (EW1S). Boston, MA: Author. Retrieved January 10, 
2014, from http://www.doe.mass.edu/edwin/analytics/ewis.html 

Mclnerney, M., & Elledge, A. (2013). Using a response to intervention framework to improve 
student learning: A pocket guide for state and district leaders. Washington, DC: Amer- 
ican Institutes for Research. Retrieved January 10, 2014, from http://www.air.org/files/ 
Response_to_Intervention_Pocket_Guide_2.pdf 

Nagaoka, J. (2013, June 6). Developing effective indicators: Lessons from the Consortium 
on Chicago School Research. In Education Northwest, REL Northwest (Webinar 
Host), Developing savvy students for college and career readiness. Part 2: Staying on 
track for college readiness (Slides 10-37) [Webinar transcript and PowerPoint slides]. 
Retrieved January 10, 2014, from http://educationnorthwest.org/resource/3537 


Ref-3 


Neild, R. C., Balfanz, R., & Herzog, L. (2007). An early warning system. Educational Lead- 
ership, 65(2), 28-33. 

Neild, R. C., & Balfanz, R. (2006). Unfulfilled promise: The dimensions and characteris- 
tics of Philadelphia’s dropout crisis, 2000-2005. Philadelphia, PA: Philadelphia Youth 
Network. http://eric.ed.gov/lkHED538341 

Robertson, M. (Producer), & Koughan, F. (Writer). (2012, July 17). Frontline: Middle 
school moment [Television broadcast]. Boston, MA: WGBH Educational Foundation. 
Retrieved April 28, 2014, from http://video.pbs.org/video/2257751072/ 

Roderick, M. (1993). The path to dropping out: Evidence for intervention. Westport, CT: Auburn 
House. 

Sparks, S. (2011, November 9). Swift growth found for “early warning” data systems. Edu- 
cation Week, 31(11), 8. 

Swanson, C. B. (2008). Cities in crisis: A special analytic report on high school graduation. 
Bethesda, MD: Editorial Projects in Education. http://eric.ed.gov/lkRED505655 

Tennessee Department of Education. (2012). Early Warning Data System: Pilot training. 
Nashville, TN: Author. Retrieved January 10, 2014, from http://www.tn.gov/education/ 
dataquality/doc/EWDS_Pilot_Training.pdf 

Therriault, S., Heppen, J., O’Cummings, M., Fryer, L., & Johnson, A. (2010). Early Warning 
System implementation guide: For use with the National High School Center’s Early Warning 
System Tool v2.0. Washington, DC: American Institutes for Research, National High 
School Center. Retrieved January 10, 2014, from http://eric.ed.gov/lkRED521686 

Therriault, S. B., O’Cummings, M., Heppen, J., Yerhot, L., & Scala, J. (2013). High School Early 
Warning Intervention Monitoring System implementation guide. Washington, DC: Amerk 
can Institutes for Research, National High School Center. http://eric.ed.gov/lkRED521686 

Uekawa, K., Merola, S., Fernandez, F., & Porowski, A. (2010). Creating an early warning 
system: Predictors of dropout in Delaware (REL MA 1.2.75-10). Rockville, MD: REL 
Mid'Atlantic. Retrieved January 10, 2014, from http://www.doe.kl2.de.us/infosuites/ 
ddoe/p20council/docs/MA1275TAFINAL508.pdf 

University of Chicago Consortium on Chicago School Research. (2007a). Freshman 
year: The make-it or break-it year [Student ed.]. Chicago, IL: Author. http://eric. 
ed.gov/lkRED498356 

University of Chicago Consortium on Chicago School Research. (2007b). Fresh- 
man year: The make-it or break-it year [Parent ed.]. Chicago, IL: Author. http://eric. 
ed.gov/?id=ED498355 

University of Chicago Consortium on Chicago School Research. (2007c). Freshman 
year: The make-it or break-it year [Teacher ed.]. Chicago, IL: Author. http://eric. 
ed.gov/?id=ED498354 


Ref-4 


University of Chicago Consortium on Chicago School Research. (2014). Selecting effective 
indicators. College Readiness Indicator Systems Resource Series. Seattle, WA: Bill & 
Melinda Gates Foundation. 

U.S. Department of Education, Institute of Education Sciences, State Longitudinal Data 
Systems Grant Program. (2012). SLDS spotlight: Development and use of early warning 
systems. Washington, DC: Author. Retrieved January 10, 2014, from http://nces.ed.gov/ 
programs/slds/pdf/Early_Warning_Systems_FINAL.pdf 

U.S. Department of Education, Institute of Education Sciences, What Works Clearing- 
house. (2006). WWC intervention report: Check & Connect. Washington, DC: Author. 
http://eric.ed.gov/lkDED525366 


Ref-5 


The Regional Educational Laboratory Program produces 7 types of reports 



Making Connections 

Studies of correlational relationships 


Making an Impact 

Studies of cause and effect 


What’s Happening 

Descriptions of policies, programs, implementation status, or data trends 


What’s Known 

Summaries of previous research 


Stated Briefly 

Summaries of research findings for specific audiences 


Applied Research Methods 

Research methods for educational settings 


Tools 

Help for planning, gathering, analyzing, or reporting data or research 



